chore: backfill email-domain member organization dates (CM-1107)#4064
chore: backfill email-domain member organization dates (CM-1107)#4064
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a one-off backfill script to infer and persist missing dateStart/dateEnd for existing email-domain memberOrganizations, using activity evidence and existing stint-inference logic, and queues affected members for affiliation recalculation.
Changes:
- Add DAL query helpers + types to find
email-domainmember orgs missing dates and derive per-org activity dates. - Add a backend script to run the backfill in batches and enqueue impacted members in Redis.
- Rename the Redis set key used by the script executor worker workflow that recalculates member affiliations.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| services/libs/data-access-layer/src/members/types.ts | Adds a typed result shape for email-domain org activity date rows. |
| services/libs/data-access-layer/src/members/organizations.ts | Adds queries for (1) memberIds with undated email-domain orgs and (2) org activity dates for stint inference. |
| services/apps/script_executor_worker/src/activities/block-project-organization-affiliations.ts | Renames the Redis set key used to queue/pop members for affiliation recalculation. |
| backend/src/bin/scripts/backfill-email-domain-member-organization-dates.ts | New one-off backfill script: batch processing, stint inference, DB updates/inserts, Redis enqueue. |
| backend/package.json | Adds an npm script entry to run the new backfill script. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
skwowet
force-pushed
the
improve/CM-1105
branch
from
d07c84e to
4f9f391
Compare
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
…and reuse Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
skwowet
force-pushed
the
improve/CM-1105
branch
from
4f9f391 to
68a79fc
Compare
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
… in member organization job Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 0c51d12. Configure here.
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…ation updates Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 5 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Summary
Adds a one-off backfill script for existing
email-domainmember organization rows with missing dates.The script:
email-domainmember organizations wheredateStartanddateEndare null.activityRelations.memberId/platform/usernameto find activity-date evidence.Note
Medium Risk
Writes backfilled
dateStart/dateEndvalues intomemberOrganizationsand changes the Redis set name used for affiliation recalculation, so a mistake could impact historical work-experience timelines or leave members un-recalculated.Overview
Adds a one-off backfill script (
backfill-email-domain-member-organization-dates) that findsemail-domainmemberOrganizationswith missing dates, derives date evidence from verified emails +activityRelations, and applies inferred stint inserts/updates in a DB transaction (including enforcing affiliation-block policies).Introduces new data-access queries to fetch affected member IDs and per-org activity dates, and standardizes the Redis queue key used to trigger member affiliation recalculation to
recalculate-member-affiliations(updated in the script executor worker and used by the new script).Reviewed by Cursor Bugbot for commit c9fac03. Bugbot is set up for automated code reviews on this repo. Configure here.